Installation¶
Just pip install:
pip install omegaconf
If you want to try this notebook after checking out the repository be sure to run
python setup.py develop at the repository root before running this code.
xxxxxxxxxxFrom a dictionary¶
xxxxxxxxxxFrom a list¶
xxxxxxxxxxFrom a yaml file¶
xxxxxxxxxxFrom a yaml string¶
xxxxxxxxxxFrom a dot-list¶
xxxxxxxxxxFrom command line arguments¶
To parse the content of sys.arg:
xxxxxxxxxxAccess and manipulation¶
Input yaml file:
xxxxxxxxxxObject style access:¶
xxxxxxxxxxdictionary style access¶
xxxxxxxxxxitems in list¶
xxxxxxxxxxChanging existing keys¶
xxxxxxxxxxAdding new keys¶
xxxxxxxxxxAdding a new dictionary¶
xxxxxxxxxxproviding default values¶
xxxxxxxxxxAccessing mandatory values¶
Accessing fields with the value ??? will cause a MissingMandatoryValue exception. Use this to indicate that the value must be set before accessing.
xxxxxxxxxxThe interpolated variable can be the dot-path to another node in the configuration, and in that case the value will be the value of that node.
Interpolations are absolute by default. Relative interpolation are prefixed by one or more dots: The first dot denotes the level of the node itself and additional dots are going up the parent hierarchy. e.g. ${..foo} points to the foo sibling of the parent of the current node.
xxxxxxxxxxxxxxxxxxxxto_yaml() will resolve interpolations if resolve=True is passed
xxxxxxxxxxInterpolations may be nested, enabling more advanced behavior like dynamically selecting a sub-config:
xxxxxxxxxxInterpolated nodes can be any node in the config, not just leaf nodes:
xxxxxxxxxxxxxxxxxxxx## Environment variable interpolationEnvironment variable interpolation is also supported.xxxxxxxxxx## Environment variable interpolationEnvironment variable interpolation is also supported.An environment variable is always returned as a string.xxxxxxxxxx# Let's set up the environment first (only needed for this demonstration)import osos.environ['USER'] = 'omry'xxxxxxxxxx# Let's set up the environment first (only needed for this demonstration)import osos.environ['USER'] = 'omry'os.environ['USERID'] = '123456'xxxxxxxxxxHere is an example config file interpolates with the USER environment variable:xxxxxxxxxxHere is an example config file interpolating with the USER and USERID environment variables:xxxxxxxxxxconf = OmegaConf.load('../source/env_interpolation.yaml')print(OmegaConf.to_yaml(conf))xxxxxxxxxxconf = OmegaConf.load('../source/env_interpolation.yaml')print(OmegaConf.to_yaml(conf, resolve=True))xxxxxxxxxxYou can specify a default value to use in case the environment variable is not defined. The following example sets `abc123` as the the default value when `DB_PASSWORD` is not defined.xxxxxxxxxxYou can specify a default value to use in case the environment variable is not defined.This default value must be a string (with the exception of `${oc.env:VAR,null}` that returns `None` if `VAR` is not defined).The following example sets `abc123` as the the default value when `DB_PASSWORD` is not defined:xxxxxxxxxxos.environ.pop('DB_PASSWORD', None) # ensure env variable does not existcfg = OmegaConf.create({'database': {'password': '${env:DB_PASSWORD,abc123}'}})cfg.database.passwordxxxxxxxxxxos.environ.pop('DB_PASSWORD', None) # ensure env variable does not existcfg = OmegaConf.create({'database': {'password': '${oc.env:DB_PASSWORD,abc123}'}})cfg.database.passwordxxxxxxxxxx## Decoding strings with interpolationsxxxxxxxxxxYou can automatically convert a string to its corresponding type (e.g., bool, int, float, dict, list) using `oc.decode` (which can even resolve interpolations).This resolver also accepts ``None`` as input, in which case it returns ``None``.This can be useful for instance to parse environment variables:xxxxxxxxxxcfg = OmegaConf.create( { "database": { "port": '${oc.decode:${oc.env:DB_PORT}}', "nodes": '${oc.decode:${oc.env:DB_NODES,null}}', "timeout": '${oc.decode:${oc.env:DB_TIMEOUT,null}}', } })os.environ["DB_PORT"] = "3308" # integeros.environ["DB_NODES"] = "[host1, host2, host3]" # listos.environ.pop("DB_TIMEOUT", None) # unset variableprint("port (int):", repr(cfg.database.port))print("nodes (list):", repr(cfg.database.nodes))print("timeout (missing variable):", repr(cfg.database.timeout))os.environ["DB_TIMEOUT"] = "${.port}"print("timeout (interpolation):", repr(cfg.database.timeout))port (int): 3308 nodes (list): ['host1', 'host2', 'host3'] timeout (missing variable): None timeout (interpolation): 3308
xxxxxxxxxxEnvironment variables are parsed when they are recognized as valid quantities that may be evaluated (e.g., int, float, dict, list):xxxxxxxxxxcfg = OmegaConf.create({'database': {'password': '${env:DB_PASSWORD,abc123}', 'user': 'someuser', 'port': '${env:DB_PORT,3306}', 'nodes': '${env:DB_NODES,[]}'}})os.environ["DB_PORT"] = '3308' # integeros.environ["DB_NODES"] = '[host1, host2, host3]' # listos.environ["DB_PASSWORD"] = 'a%#@~{}$*&^?/<' # stringprint(repr(cfg.database.port))print(repr(cfg.database.nodes))print(repr(cfg.database.password))3308
['host1', 'host2', 'host3']
'a%#@~{}$*&^?/<'
Custom interpolations¶
You can add additional interpolation types using custom resolvers. The example below creates a resolver that adds 10 to the given value.
xxxxxxxxxxYou can take advantage of nested interpolations to perform custom operations over variables:
xxxxxxxxxxBy default a custom resolver’s output is cached, so that when it is called with the same inputs we always return the same value. This behavior may be disabled by setting use_cache=False:
xxxxxxxxxxMerging configurations¶
Merging configurations enables the creation of reusable configuration files for each logical component instead of a single config file for each variation of your task.
Machine learning experiment example:
conf = OmegaConf.merge(base_cfg, model_cfg, optimizer_cfg, dataset_cfg)
Web server configuration example:
conf = OmegaConf.merge(server_cfg, plugin1_cfg, site1_cfg, site2_cfg)
The following example creates two configs from files, and one from the cli. It then combines them into a single object. Note how the port changes to 82, and how the users lists are combined.
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx